Multiple Destination Trips For Autonomous Vehicles

ABSTRACT

Aspects of the disclosure relate to a method of managing a fleet of autonomous vehicles providing trip services. The method includes receiving information identifying an intermediate destination and a final destination for a trip. In this example, the intermediate destination is a destination where an autonomous vehicle will drop off and wait for a passenger in order to continue the trip, and the final destination is a destination where the trip ends. The method also includes determining an amount of waiting time the vehicle is likely to be waiting for the passenger at the intermediate destination, determining how a vehicle of the fleet of autonomous vehicles should spend the amount of waiting time, and sending an instruction to the vehicle, based on the determination of how the vehicle should spend the amount of waiting time.

BACKGROUND

Autonomous vehicles, for instance, vehicles that do not require a humandriver, can be used to aid in the transport of passengers or items fromone location to another. Such vehicles may operate in a fully autonomousmode where passengers may provide some initial input, such as a pickupor destination location, and the vehicle maneuvers itself to thatlocation. Thus, such vehicles may be used to provide transportationservices. Other systems which provide transportation services typicallyinclude drivers or conductors who are tasked with making decisions abouthow to operate vehicles. Such services may include some backend serversystems which can dispatch vehicles to certain locations to providetransportations services as well as provide fleet management and vehiclestaging instructions.

BRIEF SUMMARY

One aspect of the disclosure provides a method of managing a fleet ofautonomous vehicles providing trip services. The method includesreceiving, by one or more computing devices, information identifying anintermediate destination and a final destination for a trip, wherein theintermediate destination is a destination where an autonomous vehiclewill drop off and wait for a passenger to return to the vehicle in orderto continue the trip and the final destination is a destination wherethe trip ends; determining, by the one or more computing devices, anamount of waiting time the vehicle is likely to be waiting for thepassenger at the intermediate destination; determining, by the one ormore computing devices, how a vehicle of the fleet of autonomousvehicles should spend the amount of waiting time; and sending, by theone or more computing devices, an instruction to the vehicle, based onthe determination of how the vehicle should spend the amount of waitingtime.

In one example, the method also includes receiving second informationindicating how long the passenger expects to spend at the intermediatedestination, and determining how the vehicle should spend the determinedamount of waiting time is further based on the second information. Inanother example, determining the amount of waiting time is based on aplurality of factors including historical waiting time data associatedwith the intermediate destination. In another example, determining theamount of waiting time is based on a plurality of factors includinghistorical waiting time data associated with time spent by the passengerat a location of a same type as the intermediate destination. In anotherexample, determining the amount of waiting time is based on a pluralityof factors including historical waiting time data associated with timespent by other passengers at the intermediate destination, the otherpassengers having a characteristic in common with the passenger. Inanother example, determining the amount of waiting time is based on aplurality of factors including whether the trip includes picking up ordropping off a second passenger at the intermediate destination. Inanother example, determining the amount of waiting time is based on aplurality of factors including whether the trip includes picking up ordropping off of cargo at the intermediate destination. In anotherexample, determining the amount of waiting time is based on a pluralityof factors including a number of passengers for the trip. In anotherexample, determining the amount of waiting time is based on a pluralityof factors including third party data indicating time spent by otherpeople at a location associated with the intermediate destination. Inanother example, determining the amount of waiting time is based on aplurality of factors including third party data indicating time spent byother people at a location of a same type as the intermediatedestination. In another example, determining the amount of waiting timeis based on determining an amount of time for each of a plurality offactors and taking an average of any determined amounts of time. Inanother example, determining the amount of waiting time is based on aplurality of factors each associated with a weight indicating aconfidence in an estimation of an amount of time for that factor, andtaking an average of the amounts of time for the plurality of factorsusing the weights. In another example, determining how the vehicleshould spend the determined amount of waiting time is based on amonetary cost for parking the vehicle during the waiting time. Inanother example, determining how the vehicle should spend the determinedamount of waiting time is based on a hierarchy of preferences includingdifferent activities. In this example, the hierarchy of preferencesincludes a preference for addressing a maintenance need of the vehicle,the place of the preference in the hierarchy depending upon a level ofurgency of the maintenance issue. In addition or alternatively, thehierarchy of preferences includes a preference for scheduling thevehicle for another trip of a predetermined duration. In this example,the place in the hierarchy of the preference for scheduling the vehiclefor another trip is below another preference for addressing amaintenance need of the vehicle. In another example, determining how thevehicle should spend the determined amount of waiting time is based onwhether a second trip estimated to take less than a first thresholdperiod of time is available. In this example, determining how thevehicle should spend the determined amount of waiting time is based onwhether the second trip would allow the vehicle to reach theintermediate destination again, after completing the second trip, withina buffer period of the determined amount of waiting time before thepassenger is expected to return to the vehicle at the intermediatedestination. In addition, the buffer period is determined based on aconfidence in the determined amount of waiting time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of an example vehicle in accordance withan exemplary embodiment.

FIG. 2 is an example of map information in accordance with aspects ofthe disclosure.

FIG. 3 is an example external view of a vehicle in accordance withaspects of the disclosure.

FIG. 4 is a pictorial diagram of an example system in accordance with anexemplary embodiment.

FIG. 5 is a functional diagram of the system of FIG. 4 in accordancewith aspects of the disclosure.

FIG. 6 is an example bird's eye view of a geographic area in accordancewith aspects of the disclosure.

FIG. 7 is an example representation of data in accordance with aspectsof the disclosure.

FIG. 8 is an example representation of data in accordance with aspectsof the disclosure.

FIG. 9 is an example representation of data in accordance with aspectsof the disclosure.

FIG. 10 is an example representation of data in accordance with aspectsof the disclosure.

FIG. 11 is an example flow diagram in accordance with aspects of thedisclosure.

DETAILED DESCRIPTION Overview

The technology relates to accommodating trips with multiple destinationsor “stops” for autonomous vehicles. Short stops may include those thatallow a passenger to get out and get back in and/or include picking upor dropping off an additional passenger while the vehicle waits for thepassenger to return in order to continue the trip. This is an especiallyimportant feature to be able to provide to customers of a ride serviceor ride-sharing service. Of course, having a vehicle wait for extendedperiod of time can be a waste of vehicle time which can become fairlysignificant as the number of vehicles and rides or trips serviced by afleet of autonomous vehicles increases.

In order to facilitate multiple destination trips, or short stops, apassenger, either before or after entering a vehicle may identify afinal destination as well as one or more intermediate destinations. Inaddition, in some instances, the passenger may also specify how long thepassenger expects to spend at each of the one or more intermediatedestinations. This information may be sent to one or more servercomputing devices of a fleet management system for managing a fleet ofautonomous vehicles.

For each of these one or more intermediate destinations, the fleetmanagement may determine how long the vehicle is likely to be waitingfor the passenger to return in order to continue the trip. This amountof waiting time may be based on the information indicating the same fromthe passenger, if available, and/or a number of other factors. Once anamount of time for an intermediate destination has been determined, thefleet management system may determine how the vehicle should spend thatamount of time. In order to do so, the fleet management system mayattempt to optimize for the vehicle servicing other trips. This may alsorequire the vehicle staying in close proximity to the intermediatedestination such that the vehicle is likely to be close by when thepassenger is actually ready to be picked up. For instance, the fleetmanagement system may follow a hierarchy of preferences for what avehicle should do. As one example, the hierarchy of preferences maystart by having the fleet management system attempt to resolve anyoutstanding urgent maintenance needs for the vehicle. Once any urgentmaintenance needs are addressed, the fleet management system maydetermine whether the vehicle is able to be assigned to a short durationtrip. If the maintenance required is less urgent, for instance aperiodic cleaning, the fleet management system may simply determinewhether the vehicle is able to be assigned to a short duration triprather than assigning the vehicle to the maintenance location.

If a short duration trip, is being requested by another passenger in thearea and the amount of time for the intermediate destination is at leastsome threshold amount of time greater than an estimated time for theshort duration trip, the fleet management system may assign the vehicleto the short duration trip. The threshold amount of time may bedetermined based on an amount of time for the vehicle to return to theintermediate destination after the short duration trip has ended plus anadditional buffer period of time. During this short duration trip and/orafterwards, the fleet management system may continuously attempt toassign the vehicle to future short duration trips if possible given thetime remaining in the determined amount of time after the short durationtrip is completed.

Of course, short duration trips may not always be available. If not, thefleet management system may determine whether a longer duration trip isbeing requested by another passenger in the area and the amount of timefor the intermediate destination is at least some threshold amount oftime greater than an estimated time for the long duration trip. If so,the fleet management system may assign the vehicle to the long durationtrip. Again, this threshold amount of time may be determined based on anamount of time for the vehicle to return to the intermediate destinationafter the long duration trip has ended plus an additional buffer periodof time. During this long duration trip and/or afterwards, the fleetmanagement system may continuously attempt follow the hierarchy ofpreferences in order to resolve maintenance needs as well as assign thevehicle to future short or long duration trips if possible given thetime remaining in the determined amount of time after the long durationtrip is estimated to be completed.

After each short or long duration trip and/or a short or long durationtrip is not available, the fleet management system may again determinewhether the vehicle needs some type of maintenance. When the maintenancehas been completed and/or if no maintenance is required, the fleetmanagement system may send an instruction to the vehicle to park andwait for the passenger.

Once the passenger is ready to be picked up at the intermediatedestination, he or she may use her client computing device to “recall”the vehicle. This may include tapping a button in an application for theride service which causes the client computing device to send acorresponding request to the fleet management system. In response, ifthe vehicle is available, the fleet management system may sendinstruction to the vehicle to return to the intermediate destination,pick up the passenger, and complete the trip.

The features described here allow an autonomous vehicle service toaccommodate trips with multiple destinations. In addition, the fleetmanagement system are able to determine what a vehicle should do whilewaiting for a passenger at an intermediate destination in a way whichallows the fleet management system to reduce the amount of time that avehicle is spent waiting in an unproductive way. In addition, what thevehicle does may be selected in order to optimize the number of tripsserviced while a vehicle would otherwise be waiting and/or for thevehicle to be able to immediately return to pick up a passenger at anintermediate destination when the passenger is ready to be picked up.The features described herein may also allow a passenger to act as apseudo dispatcher for a vehicle which they may not own, but can be usedto pick up and drop of the passenger and/or third parties at thepassenger's discretion, without having to worry about a driver or whatthe vehicle will do when the passenger is at an intermediatedestination. Moreover the features described herein may allow forvarious ownership models which can allow a passenger to effectively takea vehicle in an out of service based on how long a passenger will notrequire the vehicle.

Example Systems

As shown in FIG. 1, a vehicle 100 in accordance with one aspect of thedisclosure includes various components. While certain aspects of thedisclosure are particularly useful in connection with specific types ofvehicles, the vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, buses, recreational vehicles,etc. The vehicle may have one or more computing devices, such ascomputing devices 110 containing one or more processors 120, memory 130and other components typically present in general purpose computingdevices.

The memory 130 stores information accessible by the one or moreprocessors 120, including instructions 134 and data 132 that may beexecuted or otherwise used by the processor 120. The memory 130 may beof any type capable of storing information accessible by the processor,including a computing device-readable medium, or other medium thatstores data that may be read with the aid of an electronic device, suchas a hard-drive, memory card, ROM, RAM, DVD or other optical disks, aswell as other write-capable and read-only memories. Systems and methodsmay include different combinations of the foregoing, whereby differentportions of the instructions and data are stored on different types ofmedia.

The instructions 134 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computingdevice code on the computing device-readable medium. In that regard, theterms “instructions” and “programs” may be used interchangeably herein.The instructions may be stored in object code format for directprocessing by the processor, or in any other computing device languageincluding scripts or collections of independent source code modules thatare interpreted on demand or compiled in advance. Functions, methods androutines of the instructions are explained in more detail below.

The data 132 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 134. For instance, although the claimedsubject matter is not limited by any particular data structure, the datamay be stored in computing device registers, in a relational database asa table having a plurality of different fields and records, XMLdocuments or flat files. The data may also be formatted in any computingdevice-readable format.

The one or more processor 120 may be any conventional processors, suchas commercially available CPUs. Alternatively, the one or moreprocessors may be a dedicated device such as an ASIC or otherhardware-based processor. Although FIG. 1 functionally illustrates theprocessor, memory, and other elements of computing devices 110 as beingwithin the same block, it will be understood by those of ordinary skillin the art that the processor, computing device, or memory may actuallyinclude multiple processors, computing devices, or memories that may ormay not be stored within the same physical housing. For example, memorymay be a hard drive or other storage media located in a housingdifferent from that of computing devices 110. Accordingly, references toa processor or computing device will be understood to include referencesto a collection of processors or computing devices or memories that mayor may not operate in parallel.

Computing devices 110 may all of the components normally used inconnection with a computing device such as the processor and memorydescribed above as well as a user input 150 (e.g., a mouse, keyboard,touch screen and/or microphone) and various electronic displays (e.g., amonitor having a screen or any other electrical device that is operableto display information). In this example, the vehicle includes aninternal electronic display 152 as well as one or more speakers 154 toprovide information or audio visual experiences. In this regard,internal electronic display 152 may be located within a cabin of vehicle100 and may be used by computing devices 110 to provide information topassengers within the vehicle 100.

Computing devices 110 may also include one or more wireless networkconnections 156 to facilitate communication with other computingdevices, such as the client computing devices and server computingdevices described in detail below. The wireless network connections mayinclude short range communication protocols such as Bluetooth, Bluetoothlow energy (LE), cellular connections, as well as various configurationsand protocols including the Internet, World Wide Web, intranets, virtualprivate networks, wide area networks, local networks, private networksusing communication protocols proprietary to one or more companies,Ethernet, WiFi and HTTP, and various combinations of the foregoing.

In one example, computing devices 110 may be control computing devicesof an autonomous driving computing system or incorporated into vehicle100. The autonomous driving computing system may capable ofcommunicating with various components of the vehicle in order to controlthe movement of vehicle 100 according to primary vehicle control code ofmemory 130. For example, returning to FIG. 1, computing devices 110 maybe in communication with various systems of vehicle 100, such asdeceleration system 160, acceleration system 162, steering system 164,signaling system 166, navigation system 168, positioning system 170,perception system 172, and power system 174 (i.e. the vehicle's engineor motor) in order to control the movement, speed, etc. of vehicle 100in accordance with the instructions 134 of memory 130. Again, althoughthese systems are shown as external to computing devices 110, inactuality, these systems may also be incorporated into computing devices110, again as an autonomous driving computing system for controllingvehicle 100.

As an example, computing devices 110 may interact with one or moreactuators of the deceleration system 160 and/or acceleration system 162,such as brakes, accelerator pedal, and/or the engine or motor of thevehicle, in order to control the speed of the vehicle. Similarly, one ormore actuators of the steering system 164, such as a steering wheel,steering shaft, and/or pinion and rack in a rack and pinion system, maybe used by computing devices 110 in order to control the direction ofvehicle 100. For example, if vehicle 100 is configured for use on aroad, such as a car or truck, the steering system may include one ormore actuators to control the angle of wheels to turn the vehicle.Signaling system 166 may be used by computing devices 110 in order tosignal the vehicle's intent to other drivers or vehicles, for example,by lighting turn signals or brake lights when needed.

Navigation system 168 may be used by computing devices 110 in order todetermine and follow a route to a location. In this regard, thenavigation system 168 and/or data 132 may store detailed mapinformation, e.g., highly detailed maps identifying the shape andelevation of roadways, lane lines, intersections, crosswalks, speedlimits, traffic signals, buildings, signs, real time trafficinformation, vegetation, or other such objects and information.

FIG. 2 is an example of map information 200 for a section of roadwayincluding intersections 202 and 204. In this example, the mapinformation 200 includes information identifying the shape, location,and other characteristics of lane lines 210, 212, 214, traffic signallights 220, 222, crosswalk 230, sidewalks 240, stop signs 250, 252, andyield sign 260. Areas where the vehicle can drive may be associated withone or more rails 270, 272, and 274 which indicate the location anddirection in which a vehicle should generally travel at variouslocations in the map information. For example, a vehicle may follow rail270 when driving in the lane between lane lines 210 and 212, and maytransition to rail 272 in order to make a right turn at intersection204. Thereafter the vehicle may follow rail 274. Of course, given thenumber and nature of the rails only a few are depicted in mapinformation 200 for simplicity and ease of understanding.

Although the map information is depicted herein as an image-based map,the map information need not be entirely image based (for example,raster). For example, the map information may include one or moreroadgraphs or graph networks of information such as roads, lanes,intersections, and the connections between these features. Each featuremay be stored as graph data and may be associated with information suchas a geographic location and whether or not it is linked to otherrelated features, for example, a stop sign may be linked to a road andan intersection, etc. In some examples, the associated data may includegrid-based indices of a roadgraph to allow for efficient lookup ofcertain roadgraph features.

Positioning system 170 may be used by computing devices 110 in order todetermine the vehicle's relative or absolute position on a map or on theearth. For example, the position system 170 may include a GPS receiverto determine the device's latitude, longitude and/or altitude position.Other location systems such as laser-based localization systems,inertial-aided GPS, or camera-based localization may also be used toidentify the location of the vehicle. The location of the vehicle mayinclude an absolute geographical location, such as latitude, longitude,and altitude as well as relative location information, such as locationrelative to other cars immediately around it which can often bedetermined with less noise that absolute geographical location.

The positioning system 170 may also include other devices incommunication with computing devices 110, such as an accelerometer,gyroscope or another direction/speed detection device to determine thedirection and speed of the vehicle or changes thereto. By way of exampleonly, an acceleration device may determine its pitch, yaw or roll (orchanges thereto) relative to the direction of gravity or a planeperpendicular thereto. The device may also track increases or decreasesin speed and the direction of such changes. The device's provision oflocation and orientation data as set forth herein may be providedautomatically to the computing devices 110, other computing devices andcombinations of the foregoing.

The perception system 172 also includes one or more components fordetecting objects external to the vehicle such as other vehicles,obstacles in the roadway, traffic signals, signs, trees, etc. Forexample, the perception system 172 may include lasers, sonar, radar,cameras and/or any other detection devices that record data which may beprocessed by computing device 110. In the case where the vehicle is apassenger vehicle such as a minivan, the minivan may include a laser orother sensors mounted on the roof or other convenient location. Forinstance, FIG. 3 is an example external view of vehicle 100. In thisexample, roof-top housing 310 and dome housing 312 may include a lidarsensor as well as various cameras and radar units. In addition, housing320 located at the front end of vehicle 100 and housings 330, 332 on thedriver's and passenger's sides of the vehicle may each store a lidarsensor. For example, housing 330 is located in front of driver door 360.Vehicle 100 also includes housings 340, 342 for radar units and/orcameras also located on the roof of vehicle 100. Additional radar unitsand cameras (not shown) may be located at the front and rear ends ofvehicle 100 and/or on other positions along the roof or roof-top housing310.

The computing devices 110 may control the direction and speed of thevehicle by controlling various components. By way of example, computingdevices 110 may navigate the vehicle to a destination locationcompletely autonomously using data from the detailed map information andnavigation system 168. Computing devices 110 may use the positioningsystem 170 to determine the vehicle's location and perception system 172to detect and respond to objects when needed to reach the locationsafely. In order to do so, computing devices 110 may cause the vehicleto accelerate (e.g., by increasing fuel or other energy provided to theengine by acceleration system 162), decelerate (e.g., by decreasing thefuel supplied to the engine, changing gears, and/or by applying brakesby deceleration system 160), change direction (e.g., by turning thefront or rear wheels of vehicle 100 by steering system 164), and signalsuch changes (e.g., by lighting turn signals of signaling system 166).Thus, the acceleration system 162 and deceleration system 160 may be apart of a drivetrain that includes various components between an engineof the vehicle and the wheels of the vehicle. Again, by controllingthese systems, computing devices 110 may also control the drivetrain ofthe vehicle in order to maneuver the vehicle autonomously.

Computing device 110 of vehicle 100 may also receive or transferinformation to and from other computing devices, such as those computingdevices that are a part of the transportation service as well as othercomputing devices. FIGS. 4 and 5 are pictorial and functional diagrams,respectively, of an example system 400 that includes a plurality ofcomputing devices 410, 420, 430, 440 and a storage system 450 connectedvia a network 460. System 400 also includes vehicle 100, and vehicles100A, 100B which may be configured the same as or similarly to vehicle100. Although only a few vehicles and computing devices are depicted forsimplicity, a typical system may include significantly more.

As shown in FIG. 4, each of computing devices 410, 420, 430, 440 mayinclude one or more processors, memory, data and instructions. Suchprocessors, memories, data and instructions may be configured similarlyto one or more processors 120, memory 130, data 132, and instructions134 of computing device 110.

The network 460, and intervening nodes, may include variousconfigurations and protocols including short range communicationprotocols such as Bluetooth, Bluetooth LE, the Internet, World Wide Web,intranets, virtual private networks, wide area networks, local networks,private networks using communication protocols proprietary to one ormore companies, Ethernet, WiFi and HTTP, and various combinations of theforegoing. Such communication may be facilitated by any device capableof transmitting data to and from other computing devices, such as modemsand wireless interfaces.

In one example, one or more computing devices 110 may include one ormore server computing devices having a plurality of computing devices,e.g., a load balanced server farm, that exchange information withdifferent nodes of a network for the purpose of receiving, processingand transmitting the data to and from other computing devices. Forinstance, one or more computing devices 410 may include one or moreserver computing devices that are capable of communicating withcomputing device 110 of vehicle 100 or a similar computing device ofvehicle 100A, 100B as well as computing devices 420, 430, 440 via thenetwork 460. For example, vehicles 100, 100A, 100B may be a part of afleet of vehicles that can be dispatched by server computing devices tovarious locations. In this regard, the server computing devices 410 mayfunction as a fleet management system (hereafter, fleet managementsystem 410). In addition, as discussed further below, the vehicles ofthe fleet may periodically send the fleet management system locationinformation provided by the vehicle's respective positioning systems aswell as other information relating to the status of the vehiclesdiscussed further below, and the fleet management system may track thelocations and status of each of the vehicles of the fleet.

In addition, the fleet management system 410 may use network 460 totransmit and present information to a user, such as user 422, 432, 442on a display, such as displays 424, 434, 444 of computing devices 420,430, 440. In this regard, computing devices 420, 430, 440 may beconsidered client computing devices.

As shown in FIG. 5, each client computing device 420, 430, 440 may be apersonal computing device intended for use by a user 422, 432, 442, andhave all of the components normally used in connection with a personalcomputing device including a one or more processors (e.g., a centralprocessing unit (CPU)), memory (e.g., RAM and internal hard drives)storing data and instructions, a display such as displays 424, 434, 444(e.g., a monitor having a screen, a touch-screen, a projector, atelevision, or other device that is operable to display information),and user input devices 426, 436, 446 (e.g., a mouse, keyboard,touchscreen or microphone). The client computing devices may alsoinclude a camera for recording video streams, speakers, a networkinterface device, and all of the components used for connecting theseelements to one another.

Although the client computing devices 420, 430, and 440 may eachcomprise a full-sized personal computing device, they may alternativelycomprise mobile computing devices capable of wirelessly exchanging datawith a server computing device (such as the server computing devices ofthe fleet management system 410) over a network such as the Internet. Byway of example only, client computing device 420 may be a mobile phoneor a device such as a wireless-enabled PDA, a tablet PC, a wearablecomputing device or system, or a netbook that is capable of obtaininginformation via the Internet or other networks. In another example,client computing device 430 may be a wearable computing system, shown asa wristwatch as shown in FIG. 5. As an example the user may inputinformation using a small keyboard, a keypad, microphone, using visualsignals with a camera, or a touch screen.

As with memory 130, storage system 450 can be of any type ofcomputerized storage capable of storing information accessible by thefleet management system 410, such as a hard-drive, memory card, ROM,RAM, DVD, CD-ROM, write-capable, and read-only memories. In addition,storage system 450 may include a distributed storage system where datais stored on a plurality of different storage devices which may bephysically located at the same or different geographic locations.Storage system 450 may be connected to the computing devices via thenetwork 460 as shown in FIGS. 4 and 5, and/or may be directly connectedto or incorporated into any of the computing devices 110, 410, 420, 430,440, etc.

Storage system 450 may store various types of information as describedin more detail below. This information may be retrieved or otherwiseaccessed by a server computing device, such as those of the fleetmanagement system 410, in order to perform some or all of the featuresdescribed herein. For instance, the storage system 450 may also store ahierarchy of preferences for what a vehicle should do.

In addition, the information of storage system 450 may include thestatus and characteristics of each vehicle of the fleet, as discussedabove, as well as the map information discussed above. As the vehiclesof the fleet drive around, they may constantly and/or periodicallybroadcast to the fleet management system 410 their status. This mayinclude, for example, whether the vehicle is currently on a trip (e.g.is transporting passengers and/or cargo), a current destination of thevehicle and/or one or more additional future destinations for thevehicle (as discussed further below), whether the vehicle has anymaintenance needs, etc. As an example, maintenance needs may includewhether the vehicle needs cooling or shade, refueling or charging,cleaning, periodic inspection (after so many hours, trips, or miles inservice), recalibration of sensors (to address faults or periodicmaintenance), or other maintenance. The fleet management system 410 maystore his information in storage system 450.

In order to provide transportation services to users, the information ofstorage system 450 may include user account information such ascredentials (e.g., identifiers such as a username and password as in thecase of a traditional single-factor authentication as well as othertypes of credentials typically used in multi-factor authentications suchas random identifiers, biometrics, etc.) that can be used to identify auser to the fleet management system. The user account information mayalso include personal information such as the user's name, contactinformation, identifying information of the user's client computingdevice (or devices if multiple devices are used with the same useraccount), one or more unique signals for the user as well as other userpreference or settings data.

Example Methods

In addition to the operations described above and illustrated in thefigures, various operations will now be described. It should beunderstood that the following operations do not have to be performed inthe precise order described below. Rather, various steps can be handledin a different order or simultaneously, and steps may also be added oromitted.

As noted above, the vehicles of the fleet drive around, they mayconstantly report to the fleet management system their status. Forinstance, each vehicle's positioning system 170 may provide thevehicle's computing device 110 with the vehicle's location and position.The computing devices 110 may then send this information to the fleetmanagement system 410 as status messages. In addition to the vehicle'slocation, the different systems of the vehicle may also send informationto the computing devices 110. This may include, for example, whether thevehicle needs cooling or shade, refueling or charging, cleaning,periodic inspection (after so many hours, trips, or miles in service),recalibration of sensors (to address faults or periodic maintenance), orother maintenance. This may also be sent to the fleet management system410 in the status messages or other messages.

As noted above, the fleet management system 410 may receive the statusmessages and track the status of each vehicle in storage system 450. Forinstance, FIG. 6 depicts vehicles 100, 100A, and 110B being maneuveredon a section of roadway 600 including intersections 602 and 604. Inexample of FIG. 6, intersections 602 and 604 correspond to intersections202 and 204 of the map information 200, respectively. In this example,lane lines 610, 612, and 614 correspond to the shape, location, andother characteristics of lane lines 210, 212, and 214, respectively.Similarly, crosswalk 630 corresponds to the shape, location, and othercharacteristics of crosswalk 230, respectively; sidewalks 640 correspondto sidewalks 240; traffic signal lights 620, 622 correspond to trafficsignal lights 220, 222, respectively; stop signs 650, 652 correspond tostop signs 250, 252, respectively; and yield sign 660 corresponds toyield sign 260.

FIG. 7, is an example of status messages 710, 720, 730 for each ofvehicles 100, 100A, 100B, respectively. In this example, vehicle 100 isavailable for a trip, vehicle 100B is not available for a trip (forinstance, vehicle 100B may be occupied by passengers and/or cargo), andvehicle 100A is available for a trip. However, vehicle 100 may be“completely available” and vehicle 100A may only be available for acertain period of time. This may be determined, for instance, by thefleet management system 410 as discussed further below. In addition, themessages 710, 720, 730 also indicate that vehicles 100 and 100A do nothave maintenance needs while vehicle 100B has a maintenance need.Although not shown for simplicity, the messages may also indicate otherinformation, such as a current destination of the vehicle, a lastmessage received by the vehicle from the fleet management system, etc.

The fleet management system 410 may use these status messages to updatethe storage system 450. Thus, the fleet management system 410 may trackthe status of each vehicle over time. In addition to using the statusmessages, the fleet management system 410 may also track informationsuch as the number of hours each vehicle has been in service or out ofservice, the number of hours since the vehicle was inspected, the numberof hours since the vehicle's sensors were calibrated, and so on.

In order to request a vehicle, a user may download an application forrequesting transportation services, or rather a vehicle, to a clientcomputing device. For example, users 422 and 432 may download theapplication via a link in an email, directly from a website, or anapplication store to client computing devices 420 and 430. For example,client computing device may transmit a request for the application overthe network, for example, to one or more fleet management system 410110, and in response, receive the application. The application may beinstalled locally at the client computing device.

The user may then use his or her client computing device to access theapplication and request a vehicle. As an example, a user such as user432 may use client computing device 430 to send a request to fleetmanagement system 410 for a vehicle. As part of this, the user mayidentify a pickup location and a destination location. In order tofacilitate multiple destination trips, or short stops, a passenger,either before or after entering a vehicle, the user (now considered apassenger), may identify a final destination as well as one or moreintermediate destinations.

For instance, FIG. 8 provides an example of “current” requests 810, 820,830 for vehicles at some point close in time to the example of FIGS. 6and 7. In this example, each request may identify a passenger (forinstance, by identification number), a pickup location, and adestination location. In the examples of request 810 and 830, thepassengers have also identified one or more intermediate destinations.

FIG. 9 provides an example of location for a trip 900 for a passenger,such as the passenger who made request 810. Trip 900 includes a pick uplocation 910 for the passenger corresponding to location A (this may befor example, the current location of the passenger and/or thepassenger's client computing device), an intermediate destination 920for the passenger corresponding to location A′, and a final destination930 for the passenger corresponding to location A″. This information mayhave been provided to the fleet management system 410 by the passenger'sclient computing device, for instance, via network 460.

In response to a request for a vehicle, the fleet management system 410may select a vehicle for the passenger. For instance, the fleetmanagement system 410 may select an available vehicle, such as vehicle100, to be assigned to the passenger of request 810 based on theinformation in the storage system 450 regarding the status of eachvehicle of the fleet of vehicles. For instance, a vehicle may beselected if it is nearby the passenger, is free to providetransportation services, is not in need of urgent maintenance, etc.

Once selected, the available vehicle may be assigned to the passenger.For instance, the fleet management system 410 may update the status ofvehicle 100 in the storage system 450. In addition, the fleet managementsystem 410 may send the pickup location to the computing devices 110 ofvehicle 100, for instance, via network 460. The computing devices 110may also set the pickup location as the “next destination” for thevehicle 100. In addition, the computing devices 110 may also receive thefinal destination as well as the intermediate destination, and/or thisinformation may be sent to the vehicle's computing devices when thevehicle reaches the pickup location and/or the intermediate destination.

In addition, in some instances, the passenger may also specify how longthe passenger expects to spend at each of the one or more intermediatedestinations. For instance, the passenger may specify an expected amountof time indicating how long he or she expects to be at a location ofintermediate destination 920. This may occur when the passengerspecifies the one or more intermediate destinations, when the vehiclereaches each of the one or more intermediate destinations, or at somepoint there between. In this regard, this information may be entered bythe passenger into the passenger's client computing device (forinstance, via the application) and/or via a user input of the vehicle,such as user input 150. This information may be sent by the passenger'sclient computing device and/or the computing devices 110 to the fleetmanagement system 410, again via network 460.

For each of these one or more intermediate destinations, the fleetmanagement system 410 may determine how long the vehicle is likely to bewaiting for the passenger to return in order to continue the trip. Thisamount of waiting time may be based on an expected amount of timeprovided by the passenger, if available, and/or a plurality of factors.The plurality of factors may include, for instance, historical waitingtimes, the number of passengers, whether the intermediate destination ismerely to pick up or drop off another passenger (e.g., where the ridewill continue with one or more other passengers), data from third partysources about how long people (e.g., customers) tend to stay at alocation associated with the intermediate destination or other similarlocation(s) (e.g. a location of the same or similar type), particularlyat or around a time of day when the vehicle will be stopped at theintermediate destination (e.g., a grocery store is typically less busyat 11 pm than at 11 am, the amount of time a passenger may spend at aservice provider such as a nail salon, hospital, clinic, dental office,etc., may be longer than the amount of time the passenger would spend ata bank or a post office, the amount of time a passenger may spend at apost office may vary depending on the time of day, month or season, andso on), whether the passenger and/or a third party will have to load orunload cargo at the intermediate destination, real time trafficcongestion or business of the surroundings (e.g., it could take longerfor a passenger to reach the vehicle in a busy parking lot as comparedto a less busy parking lot), and more. As examples, historical waitingtimes may include wait times of the past history of this particularpassenger at this particular location or other similar location(s), thepast history of similarly situated passengers (i.e., other passengerswith the same or similar demographics or characteristics (e.g.,infirmity, age, profession, gender, etc.), traveling with children, etc.as the passenger) at this particular location or other similarlocation(s), past history of all passengers at this particular locationor other similar location(s), etc. Real time traffic congestion orbusiness may be obtained from informational services (e.g., third partysources), for instance, by one or more other server computing devicesvia network 460, from camera or video feeds of the area, etc. In someinstances, multiple locations may be associated with an intermediatedestination (e.g., a group of shops or service providers, or a clusterof office buildings, sharing a driveway or parking lot), and a passengermay indicate one or more of the locations that the passenger intends tovisit at the intermediate destination, which may then be factored intothe waiting time amount. In other instances where multiple locations areassociated with an intermediate destination, a passenger may specify anexpected amount of waiting time, which further may be modifiable by thepassenger while the passenger is at the intermediate destination (e.g.,if lines are unexpectedly long or short, if the passenger chooses to addmore errands at the intermediate destination, or other reasons).

As one example, the fleet management system 410 may estimate an amountof waiting time based on each of the plurality of factors (assumingthere is information available for a given factor) and take an average.As another example, each of the plurality of factors may be weighted bya confidence in an estimation for that factor and thereafter take anaverage. Of course, the larger the standard deviation among theestimated amounts of time for the factors, the lower the confidence inthe average amount of time or the determined amount of time for theintermediate destination.

Once an amount of waiting time for an intermediate destination has beendetermined, the fleet management system 410 may determine how thevehicle should spend that amount of waiting time. In order to do so, thefleet management system may attempt to optimize for the vehicleservicing other trips. This may also require the vehicle staying inclose proximity to the intermediate destination such that the vehicle islikely to be close by when the passenger is actually ready to be pickedup. For instance, the fleet management system 410 may follow theaforementioned hierarchy of preferences for what a vehicle should dostored in storage system 450.

The hierarchy of preferences may start by having the fleet managementsystem 410 attempt to resolve any outstanding urgent maintenance needsfor the vehicle. For instance, the status of the vehicle 100 with regardto maintenance needs may change over time. In this regard, the vehicle100, though it did not have maintenance needs at the time of the exampleof FIG. 7, this may no longer be the case once the vehicle is proximateto the intermediate destination. In addition, different maintenanceneeds may be associated with different levels of urgency. Higher levelsor urgency may be correspond urgent maintenance needs which cannot bedeferred till a later time, and lower levels of urgency may correspondto less urgent or non-urgent maintenance needs which can be deferredtill a later time. If any urgent maintenance needs are resolved and/orcan be delayed in resolving, the hierarchy of preferences may then havethe fleet management system 410 send the vehicle on one or more shortduration trips which do not take the vehicle more than somepredetermined distance in time or space from the location of theadditional location. If not available or possible given the amount ofwaiting time, the hierarchy of preferences may have the fleet managementsystem 410 attempt to send the vehicle on one or more long durationtrips which do not take the vehicle more than some predetermineddistance in time or space from the location of the additional location.If not available or possible given the determined amount of waitingtime, the hierarchy of preferences may have the fleet management system410 again determine whether the vehicle requires maintenance, and ifthere is enough time, resolve any less urgent maintenance needs.Thereafter, the hierarchy of preferences may cause the fleet managementsystem 410 provide the vehicle with instructions to park and wait.

For instance, the fleet management system 410 may first determinewhether the vehicle needs some type of maintenance. This may bedetermined, for instance, by referring to the status of the vehicle 100in the storage system 450. If the vehicle does require maintenance, andthe maintenance required is urgent (i.e. there is a problem with acritical sensor or the vehicle is running out of fuel and/or electriccharge), the fleet management system 410 may next determine whether amaintenance and/or charging location is nearby to the intermediatedestination. If so, the fleet management system 410 may assign thevehicle to the maintenance location until the maintenance is completed.In this regard, the vehicle is able to be productive when the vehiclewould otherwise be waiting for the passenger to finish at theintermediate destination. The fleet management system 410 110 may sendinstructions to the vehicle, for instance via network 460, to cause thevehicle to proceed to the maintenance location from the intermediatedestination. Again, this may cause the computing devices of the vehicleto set the vehicle's current destination to the maintenance location.This may also be reported back to the fleet management system in thenext status message from the vehicle.

Similarly, if the maintenance was previously scheduled and the vehiclehas a window during which it can go to a maintenance location, the fleetmanagement system 410 may assign the vehicle to the maintenance locationuntil the maintenance is completed. In another example, if a vehiclerequires a charge and there is an opportunity to recharge the vehicle ata nearby high speed charging station and/or the current cost ofelectricity is low (e.g. charging costs may be lower at different timesof day), the fleet management system 410 may assign the vehicle tocharging location until the maintenance is completed. In this regard,the vehicle is able to be productive when the vehicle would otherwise bewaiting for the passenger to finish at the intermediate destination.Again, the fleet management system 410 may send instructions to thevehicle's computing devices, for instance via network 460, to cause thevehicle to proceed to the maintenance location. Of course, if thecharging costs are high and the vehicle is able to service one or moreshort trips, the fleet management system 410 may defer assigning thevehicle to a charging location.

Once any urgent maintenance is addressed, the fleet management system410 may determine whether the vehicle is able to be assigned to a shortduration trip. In other words, non-urgent maintenance, for instance aperiodic cleaning, may be differed in favor of a suitable short durationtrip. In such cases, the fleet management system 410 may simplydetermine whether the vehicle is able to be assigned to a suitable shortduration trip rather than assigning the vehicle to the maintenancelocation. For instance, a short duration trip may be considered suitableif the amount of time for the vehicle to complete the short durationtrip a is at least some threshold amount of time less than thedetermined amount of waiting time. If so, the fleet management system410 may send an instruction to the vehicle, for instance via network460, to proceed to perform the short duration trip. As an example, ashort duration trip may be a trip that is less than some predeterminedperiod of time, such as 10 minutes or more or less.

For instance, FIG. 10 provides an example of a “current” request 1010for a vehicle at some point later in time before or after the vehicle100 drops off the passenger at the intermediate destination 920. In thisexample, each request may identify a passenger (for instance, byidentification number), a pickup location, and a destination location.For instance, if request 1010 is a short duration trip, the vehicle 100may be assigned to the passenger of request 1010. In this regard, thevehicle is able to be productive when the vehicle would otherwise bewaiting for the passenger to finish at the intermediate destination. Thefleet management system 410 may send instructions to the vehicle, forinstance via network 460, to cause the vehicle to proceed to perform theshort duration trip. Examples of short duration trips may include tripsto transport passengers and/or cargo such as an errand, delivery, orpick up for either the passenger or another person could be done duringthis wait time (e.g. picking up dry cleaning, coffee, or items at astore, etc.).

The threshold amount of time may be determined based on an amount oftime for the vehicle to return to the intermediate destination after theshort duration trip has ended plus an additional buffer period of time.This buffer period of time may be a fixed period of time, such as 5minutes or more or less, or may be determined based on the confidence inthe determined amount of time. In other words, a larger buffer may beused when the confidence is low, and a smaller buffer may be used whenthe confidence is high. During this short duration trip and/orafterwards, the fleet management system 410 may continuously attempt toassign the vehicle to future short duration trips if possible given thetime remaining in the determined amount of time after the short durationtrip is completed.

Of course, short duration trips may not always be available. If not, thefleet management system 410 may determine whether a suitable longduration trip is being requested by another passenger in the area. Along duration trip may be a trip that is greater than some predeterminedperiod of time, such as 10 minutes or more or less. In addition, a longduration trip may be considered suitable if the amount of time for thevehicle to complete the long duration trip is at least some thresholdamount of time less than the determined amount of waiting time. Ofcourse, whether a vehicle can service a long duration trip may also bedetermined by the fleet management system 410 based on the status of thevehicle (e.g. whether the vehicle has enough fuel or battery charge). Ifso, the fleet management system 410 may assign the vehicle to the longduration trip. For instance, returning to FIG. 10, if request 1010 is along duration trip, the vehicle 100 may be assigned to the passenger ofrequest 1010. In this regard, the vehicle is able to be productive whenthe vehicle would otherwise be waiting for the passenger to finish atthe intermediate destination. Again, the fleet management system 410 110may send instructions to the vehicle, for instance via network 460, tocause the vehicle to proceed to perform the long duration trip.

Again, this threshold amount of time may be determined based on anamount of time for the vehicle to return to the intermediate destinationafter the long duration trip has ended plus an additional buffer periodof time. In addition, this buffer period of time may be a fixed periodof time, such as 5 minutes or more or less, or may be determined basedon the confidence in the determined amount of time. In other words, alarger buffer may be used when the confidence is low, and a smallerbuffer may be used when the confidence is high. During this longduration trip and/or afterwards, the fleet management system 410 maycontinuously attempt follow the hierarchy of preferences in order toresolve maintenance needs as well as assign the vehicle to future shortor long duration trips if possible given the time remaining in thedetermined amount of time after the long duration trip is estimated tobe completed.

After each short or long duration trip and/or in the event a short orlong duration trip is not available, the fleet management system 410 mayagain proceed to follow the hierarchy of preferences. This may involvethe fleet management system again determining whether the vehicle needssome type of maintenance. If the maintenance required is urgent (i.e.there is a problem with a critical sensor or the vehicle is running outof fuel) and a maintenance location is nearby, the fleet managementsystem 410 may assign the vehicle to the maintenance location until themaintenance is completed. If the maintenance required is less urgent,the fleet management system 410 may determine whether the vehicle isable to be assigned to a maintenance location and have the maintenancecompleted given the time remaining in the determined amount of timeafter the maintenance is estimated to be completed. As in the otherexamples, this may require there being sufficient time remaining suchthat the vehicle is able to return to the intermediate destination afterthe maintenance has ended plus an additional buffer period of time.Again, this buffer period of time may be a fixed period of time, such as5 minutes or more or less, or may be determined based on the confidencein the determined amount of time.

When the maintenance has been completed and/or if no maintenance isrequired and there are no short or long duration trips suitable for thevehicle, the fleet management system 410 may send an instruction to thevehicle, for instance via network 460, to cause the vehicle to park andwait for the passenger at the intermediate destination. In someinstances, the fleet management system 410 may send instructions for thevehicle to park nearby to the location of the intermediate destination,especially if the determined amount of waiting time is very short. Ifthe determined amount of waiting time is longer and/or there are costs(e.g. paid parking) involved with parking close to the location of theintermediate destination, the fleet management system 410 may sendinstructions for the vehicle to park farther away from the location ofthe intermediate destination. In some instances, even where there areparking costs, the fleet management system 410 may send instructions topark nearby based on a preference of the passenger (for instance, afterprompting the passenger to confirm that he or she is willing to pay orprepay for the parking costs and/or other costs resulting from losttrips due to waiting for the passenger) and/or whether the passenger hasreserved the vehicle for a particular level of service as discussedfurther below.

For instance, the hierarchy of preferences may be adjusted or adifferent hierarchy of preferences may be used depending upon the levelof service. In other words, the aforementioned hierarchy of preferencesmay be a “default” hierarchy of preferences for a default level ofservice. If a passenger chooses to opt for a higher level of service thefleet management system 410 may optimize for the vehicle to be availableby when the passenger is actually ready to be picked up. For instance,the hierarchy of preferences may be changed from the default in variousways such as not allowing the fleet management system 410 to assign longduration trips, decreasing the period defining short duration trips ornot allowing the fleet management system 410 to assign short durationtrips at all, increasing the aforementioned buffer for short durationtrips, causing the vehicle to park closer to the location of theintermediate destination (even when this requires payment or additionalfees to park), etc.

Once the passenger is ready to be picked up at the intermediatedestination, he or she may use her client computing device to “recall”the vehicle. This may include tapping a button or otherwise selecting anoption in the application using the passenger's client computing devicewhich causes the client computing device to send a request to the fleetmanagement system 410 to recall the vehicle. In response, if the vehicle100 is available (i.e. not getting maintenance or servicing a trip), thefleet management system 410 may send instruction to the vehicle, forinstance via network 460, to return to the intermediate destination,pick up the passenger, and complete the trip. This may include stoppingat additional intermediate destinations as well as transporting thepassenger to the final destination. If the vehicle 100 is not available,the fleet management system 410 may estimate a period of time for thevehicle to reach the intermediate destination. If this period of time isless than a threshold period of time, for instance 10 minutes or more orless (which of course, may be adjusted downward depending upon the levelof service), the fleet management system 410 may provide the passengerwith options for waiting getting another vehicle, for instance via theapplication on the passenger's client computing device.

In some instance, the passenger may have left articles in the vehicle.If so, the passenger may be given an option in the application via thepassenger's client computing device to have those articles delivered tothe passenger at a particular location within a service area of thevehicle sometime later. Of course, there may be limits on this, such asa reasonable time length at which point the goods may be delivered ordeposited to a predetermined location, such as the passenger's home, thepassenger's work, a location also managed by or affiliated with theservice (i.e. a vehicle depot, a coffee shop or restaurant affiliatedwith the service, or other such location) convenient for the passenger,a physical locker at a particular location, or some other reasonablelocation. The delivery may be accomplished by requiring the passenger(or rather the passenger's client computing device) to authenticate tothe vehicle, speak and/or be viewed by a remote customer service agent,etc. before allowing the passenger access to the articles to protect thepassenger's articles from theft. In addition, to facilitate the abilityfor a passenger to leave articles in a vehicle safely while thepassenger is at an intermediate destination, the vehicle may includelockable compartments to safely store articles.

Although the examples above suggest determining an amount of waitingtime may occur once per trip or per intermediate destination, the amountof waiting time may actually be updated. These updates may be periodic,for instance every minute or more or less, or any time thatcircumstances may indicate that the amount of waiting time has changed.For instance, as noted above, if a passenger modifies, such as byincreasing or decreasing, an expected amount of waiting time, forexample using an option in the application on his or her clientcomputing device, the amount of waiting time may be updated orrecalculated using any of the factors described above.

In some instances, the intermediate destination may be a pick uplocation that occurs before the passenger who scheduled the trip. Inthis regard, the intermediate destination may involve picking up a thirdparty to ride with the passenger when the passenger is not in thevehicle. In order to ensure that the correct third party enters thevehicle (or places cargo in the vehicle), the fleet management system410 may also be able to implement authentication using a token (forinstance, over text message or email to the third party once identifiedby the passenger), using ultrasound (whisper net), confirming the thirdparty with a camera image or video image of the third party sent to thepassenger, a call to customer service, etc.

FIG. 11 includes an example flow diagram 1100 of some of the examplesfor managing a fleet of autonomous vehicles providing trip service asdiscussed above. In this example, the steps of flow diagram may beperformed by one or more processors of one or more computing devices,such as processors of the server computing devices of the fleetmanagement system 410. At block 1110, information identifying anintermediate destination and a final destination for a trip isidentified. In this example, the intermediate destination is adestination where an autonomous vehicle will drop off and wait for thepassenger in order to continue the trip, and the final destination is adestination where the vehicle will drop off the passenger and the tripends. At block 1120, how long the vehicle is likely to be waiting forthe passenger at the intermediate destination is determined. At block1130, how a vehicle of the fleet of autonomous vehicles should spend thedetermined amount of time is determined. An instruction is sent to thevehicle based on the determination of how the vehicle should spend thatamount of time at block 1140.

The features described here allow an autonomous vehicle service toaccommodate trips with multiple destinations. In addition, the fleetmanagement system is able to determine what a vehicle should do whilewaiting for a passenger at an intermediate destination in a way whichallows the fleet management system to reduce the amount of time that avehicle is spent waiting in an unproductive way. In addition, as notedabove, what the vehicle does may be selected in order to optimize thenumber of trips serviced while a vehicle would otherwise be waitingand/or for the vehicle to be able to immediately return to pick up apassenger at an intermediate destination when the passenger is ready tobe picked up. In addition, the features described herein allow apassenger to act as a pseudo dispatcher for a vehicle which they may notown, but can be used to pick up and drop of the passenger and/or thirdparties at the passenger's discretion, without having to worry about adriver or what the vehicle will do when the passenger is at anintermediate destination. Moreover the features described herein mayallow for various ownership models which can allow a passenger toeffectively take a vehicle in an out of service based on how long apassenger will not require the vehicle (i.e. how long the passenger willbe at the intermediate destination).

Unless otherwise stated, the foregoing alternative examples are notmutually exclusive, but may be implemented in various combinations toachieve unique advantages. As these and other variations andcombinations of the features discussed above can be utilized withoutdeparting from the subject matter defined by the claims, the foregoingdescription of the embodiments should be taken by way of illustrationrather than by way of limitation of the subject matter defined by theclaims. In addition, the provision of the examples described herein, aswell as clauses phrased as “such as,” “including” and the like, shouldnot be interpreted as limiting the subject matter of the claims to thespecific examples; rather, the examples are intended to illustrate onlyone of many possible embodiments. Further, the same reference numbers indifferent drawings can identify the same or similar elements.

1. A method of managing a fleet of autonomous vehicles providing tripservices, the method comprising: receiving, by one or more computingdevices, information identifying an intermediate destination and a finaldestination for a trip, wherein the intermediate destination is adestination where an autonomous vehicle will drop off and wait for apassenger to return to the vehicle in order to continue the trip and thefinal destination is a destination where the trip ends; determining, bythe one or more computing devices, an amount of waiting time the vehicleis likely to be waiting for the passenger at the intermediatedestination; determining, by the one or more computing devices, how avehicle of the fleet of autonomous vehicles should spend the amount ofwaiting time; and sending, by the one or more computing devices, aninstruction to the vehicle, based on the determination of how thevehicle should spend the amount of waiting time.
 2. The method of claim1, further comprising, receiving second information indicating how longthe passenger expects to spend at the intermediate destination, andwherein determining how the vehicle should spend the determined amountof waiting time is further based on the second information.
 3. Themethod of claim 1, wherein determining the amount of waiting time isbased on a plurality of factors including historical waiting time dataassociated with the intermediate destination.
 4. The method of claim 1,wherein determining the amount of waiting time is based on a pluralityof factors including historical waiting time data associated with timespent by the passenger at a location of a same type as the intermediatedestination.
 5. The method of claim 1, wherein determining the amount ofwaiting time is based on a plurality of factors including historicalwaiting time data associated with time spent by other passengers at theintermediate destination, the other passengers having a characteristicin common with the passenger.
 6. The method of claim 1, whereindetermining the amount of waiting time is based on a plurality offactors including whether the trip includes picking up or dropping off asecond passenger at the intermediate destination.
 7. The method of claim1, wherein determining the amount of waiting time is based on aplurality of factors including whether the trip includes picking up ordropping off of cargo at the intermediate destination.
 8. The method ofclaim 1, wherein determining the amount of waiting time is based on aplurality of factors including a number of passengers for the trip. 9.The method of claim 1, wherein determining the amount of waiting time isbased on a plurality of factors including third party data indicatingtime spent by other people at a location associated with theintermediate destination.
 10. The method of claim 1, wherein determiningthe amount of waiting time is based on a plurality of factors includingthird party data indicating time spent by other people at a location ofa same type as the intermediate destination.
 11. The method of claim 1,wherein determining the amount of waiting time is based on determiningan amount of time for each of a plurality of factors and taking anaverage of any determined amounts of time.
 12. The method of claim 1,wherein determining the amount of waiting time is based on a pluralityof factors each associated with a weight indicating a confidence in anestimation of an amount of time for that factor, and taking an averageof the amounts of time for the plurality of factors using the weights.13. The method of claim 1, wherein determining how the vehicle shouldspend the determined amount of waiting time is based on a monetary costfor parking the vehicle during the waiting time.
 14. The method of claim1, wherein determining how the vehicle should spend the determinedamount of waiting time is based on a hierarchy of preferences includingdifferent activities.
 15. The method of claim 14, wherein the hierarchyof preferences includes a preference for addressing a maintenance needof the vehicle, the place of the preference in the hierarchy dependingupon a level of urgency of the maintenance issue.
 16. The method ofclaim 14, wherein the hierarchy of preferences includes a preference forscheduling the vehicle for another trip of a predetermined duration. 17.The method of claim 16, wherein the place in the hierarchy of thepreference for scheduling the vehicle for another trip is below anotherpreference for addressing a maintenance need of the vehicle.
 18. Themethod of claim 1, wherein determining how the vehicle should spend thedetermined amount of waiting time is based on whether a second tripestimated to take less than a first threshold period of time isavailable.
 19. The method of claim 18, wherein determining how thevehicle should spend the determined amount of waiting time is based onwhether the second trip would allow the vehicle to reach theintermediate destination again, after completing the second trip, withina buffer period of the determined amount of waiting time before thepassenger is expected to return to the vehicle at the intermediatedestination.
 20. The method of claim 19, wherein the buffer period isdetermined based on a confidence in the determined amount of waitingtime.